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The technological advancements in the field of remote sensing have resulted in substantial growth of the telemedicine industry. 
While health care practitioners may now monitor their patients’ well-being from a distance and deliver their services remotely, 
the lack of physical presence introduces security risks, primarily with regard to the identity of the involved parties. The sensing 
apparatus, that a patient may employ at home, collects and transmits vital signals to medical centres which respond with treatment 
decisions despite the lack of solid authentication of the transmitter’s identity. In essence, remote monitoring increases the risks 
of identity fraud in health care. This paper proposes a biometric identification solution suitable for continuous monitoring 
environments. The system uses the electrocardiogram (ECG) signal in order to extract unique characteristics which allow to 
discriminate users. In security, ECG falls under the category of medical biometrics, a relatively young but promising field of 
biometric security solutions. In this work, the authors investigate the idiosyncratic properties of home telemonitoring that 
may affect the ECG signal and compromise security. The effects of psychological changes on the ECG waveform are taken into 
consideration for the design of a robust biometric system that can identify users based on cardiac signals despite physical or 
emotional variations. 


1. Introduction the lack of physical presence at the time of collection of the 
medical information (e.g., vital signals), the identity of the 


For a number of severe diseases such as diabetes, hyperten- user that transmits the respective information is uncertain. 


sion, or respiratory disorders, where hospitalization might 
not always be justified or needed, home care is traditionally 
preferred. Moreover, the visit of a medical practitioner to the 
patient’s home is not necessarily an efficient solution either. 
This is because (i) the high health care costs for this service 
are undesirable; (ii) it may be infeasible for the personnel 
to reach highly rural areas; and (iii) monitoring is usually 
required on a continuous basis, rather than per visit. Home 
telemonitoring is now a reality, addressing the above problem 
very effectively, that is, it is not only cost-efficient but can also 
reach isolated communities and allow for 24 hr reporting on 
the patient’s status. 


Nevertheless, the widespread utilization of telemonitor- 
ing increases the risk of identity fraud in health care. Due to 


Typically, every monitoring device is assigned with a unique 
ID (e.g., a serial code) in order to identify the transmitter. 
However, this only partially solves the problem since such a 
strategy identifies the device but not the person that utilizes 
it. What is needed is a means for authentication that is 
directly linked to the user and which can identify that person 
continuously. 

Traditional approaches to identity authentication include 
something that the user remembers (e.g., password, PIN 
number) or possesses (e.g., ID card, token). Neither of these 
two approaches can solve efficiently the problem of remote 
authentication, since they cannot be performed in a contin- 
uous manner. It is not practical to ask a user to be password 
authenticated during monitoring sessions. Furthermore, the 


above entities can be easily stolen or forgotten. Biometric 
security was introduced as an alternative to this problem. 
Linking identities with bodily characteristics that cannot be 
forgotten or easily stolen provides airtight security. 

In this work, the ECG signal is proposed for identity 
authentication in remote monitoring settings. The rationale 
behind this choice is that this signal is very likely to be 
collected routinely in such environments as it is typically 
used for medical diagnosis of several cardiac and noncardiac 
conditions. In addition, as opposed to “static” face or iris 
images, the ECG signal is time dependent which suits con- 
tinuous authentication since a fresh reading can be acquired 
every couple of seconds and be used to reauthenticate a 
particular user. 

From a security point of view, ECG falls under the cate- 
gory of medical biometrics, that is, physiological characteris- 
tics that are typically used within health care environments 
for clinical diagnoses. However, there is evidence that some 
of these vital signals such as the ECG, phonocardiogram 
(PCG), photoplethysmogram (PPG), and blood volume 
pressure (BVP) carry information which is unique for 
every individual [1-5]. With the advances in the sensing 
technology, the potential of using these signals for biometric 
recognition is great. Although this work deals with the prob- 
lem of human authentication using the ECG signal, similar 
concepts are valid for most medical biometric modalities. 

The ECG biometric provides inherent liveness detection 
which has computational advantages, since most of the 
existing modalities would require additional mechanisms to 
validate the liveness of the sensor’s reading. In addition, ECG 
is naturally immune to falsification or replay attacks, as it is 
extremely difficult to steal and inject someone’s ECG into a 
biometric system. Heart signals are universal, stable over a 
large period of time, and sufficiently unique. The intersubject 
variability comes from the fact that ECG pictures the electro- 
physiological variations of the myocardium and is affected 
by factors such as the heart mass orientation, conductivity of 
cardiac muscles and activation order [6, 7]. This variability 
has been extensively investigated in the medical research, for 
the establishment of universal diagnostic standards [8]. 

Factors that affect the ECG waveform can be classified 
as physiological or psychological. While a heart rate increase 
due to exercise decreases the ECG “period” (in reality this 
signal is quasiperiodic), emotions may also change the ECG 
waveform. While the interaction between physiological and 
psychological factors on the ECG signal is obscure, there is 
evidence that emotional activity directly impacts the ECG 
waveform [9-11]. This aspect of the ECG is very important 
when deploying it for biometric recognition, because it can 
significantly affect the overall accuracy. In welfare monitor- 
ing environments, physiological and psychological variations 
are expected and one needs to account for such effects. 

The objective of this work is to first demonstrate 
that changes in the psychological status of a subject can 
potentially affect the ECG biometric template and second to 
provide a solution to this problem for welfare monitoring 
applications. While emotional activity can compromise the 
stability and robustness of a biometric template, prior works 
have concentrated on the effects of just physical activity. In 
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this work, a template updating methodology is proposed to 
automatically adjust to the psychological status of a user in 
order to treat false rejections. 


2. The Electrocardiogram (ECG) 


The ECG is one of the most widely used signals in health care. 
Recorded at the surface of the body, with electrodes attached 
in various configurations, the ECG signal is studied for diag- 
nostics even at the very early stage of a disease. In essence, this 
signal describes the electrical activity of the heart over time 
and pictures the sequential depolarization and repolarization 
of the different muscles that form the myocardium. 

The ECG is a quasiperiodic signal, composed of succes- 
sive heart beats. Every heart beat has three essential compo- 
nents as follows: the P wave, the QRS complex, and the T 
wave. The P wave describes the depolarization of the right 
and left atria. The amplitude of this wave is relatively small, 
because the atrial muscle mass is limited. The QRS complex 
is the largest of three since it represents the depolarization 
of the right and left ventricles, which are the chambers with 
the most substantial mass in the heart. Finally, the T has 
a relatively small amplitude and it depicts the ventricular 
repolarization. 

Under normal physiological changes, that is, when the 
heart rate increases due to physical activity (tachycardia), or 
decreases due to meditative tasks (bradycardia), the relative 
position of the T wave may vary. Although it is usually 
observed about 300 ms after the QRS complex, it may appear 
closer to the QRS complex at rapid rates and further away 
at slow rates [12]. This variation is not so noticeable among 
the P wave and the QRS complex, which are more stable 
components of the heart cycle. 

While the effects of physical activity on the ECG signal 
are well established, the reactivity to psychological changes 
is more obscure. The autonomic nervous system (ANS) has 
nerve endings within the cardiac muscle which play a major 
role in the cardiac output because they affect the rhythm at 
which the muscle pumps blood. The fibers of the sympathetic 
system run along the atria and the ventricles and when 
activated stimulate the cardiac muscle to increase the heart 
rate. On the other hand, the parasympathetic system reduces 
the cardiac workload. In the presence of a mental stressor 
specifically, the sympathetic system dominates the parasym- 
pathetic, resulting in the following reactivity effects [13]. 


(1) Automaticity: the intrinsic impulse firing (automatic- 
ity) of the pacemaker cells increases, which translates 
directly to an increased heart rate. 


(2) Contractility: during every contraction the fibers 
of the heart shorten more, compared to the case 
during homeostasis, thereby increasing the force of 
contraction. 


(3) Conduction rate: the natural pacemaker, the SA node, 
is forced to conduct faster. 

(4) Excitability: during sympathetic stimulation, the per- 
son has increased perceptiveness to internal and 
external stimuli, which increases the irritability of the 
cardiac muscle and possibly lead to ectopic beats. 


Journal of Computer Networks and Communications 


(5) Dilation of coronary blood vessels: the diameter of 
the coronary blood vessels increases, followed by 
increased blood flow to the cardiac muscle. 


While there are several open questions on the exact 
effects on the ECG for every experienced emotion, the 
ability of detecting emotion from this waveform has been 
demonstrated in [9]. This work demonstrates that the effects 
of psychological activity on the ECG waveform are significant 
enough to endanger biometric accuracy. 


3. ECG in Biometric Recognition 


Prior works in the ECG biometric recognition field can be 
categorized as either fiducial points dependent or indepen- 
dent. Fiducials are specific points of interest on an ECG heart 
beat such as the onset and the offset of the heart beat waves. 
Fiducial-based approaches rely on local features of the heart 
beats for biometric template design, such as the temporal or 
amplitude difference between consecutive fiducial points. On 
the other hand, fiducial points independent approaches treat 
the ECG signal or isolated heart beats holistically and extract 
features statistically based on the overall morphology of the 
waveform. 

Both approaches have advantages and disadvantages. 
While fiducial-oriented features risk to miss identifying 
information hidden behind the overall morphology of 
the biometric modality, holistic approaches deal with a 
large amount of redundant information that needs to be 
eliminated. The challenge in the latter case is to remove this 
information in a way that the intrasubject variability is min- 
imized and the intersubject is maximized. For the ECG case, 
detecting fiducial points is a very obscure process due to the 
high variability of the signal. In fact, there is no universally 
acknowledged rule that can guide this detection [7]. 


3.1. Fiducial-Dependent Approaches. Among the earliest 
works in the area is Biel et al’s [14] proposal, in 2001, for 
a fiducial feature extraction algorithm, which demonstrated 
the feasibility of using ECG signals for human identification. 
The standard 12 lead system was used to record signals 
from 20 subjects of various ages. Kyoso and Uchiyama [15] 
proposed four fiducial-based features for ECG biometric 
recognition, that is, the P wave duration, PQ interval, QRS 
complex, and QT durations and achieved 94.2% biometric 
accuracy. 

In 2002, Shen et al. [16] reported an ECG-based recogni- 
tion method with seven fiducial-based features that relate to 
the QRS complex. The underlying idea was that this wave is 
less affected by varying heart rates and thus is appropriate for 
biometric recognition. More complete biometric recognition 
tests were reported in 2004, by Israel et al. [1]. This 
work presented the three clear stages of ECG biometric 
recognition, that is, preprocessing, feature extraction and 
classification. In addition, a variety of experimental settings 
are described in [1] such as examination of variations due to 
electrode placement and physical stress. 

Similarly, Palaniappan and Krishnan [17] used a form 
factor, which is a measure of signal complexity, and tested 
using a neural network classifier. An identification rate 


of 97.6% was achieved over recordings of 10 individuals. 
Kim et al. [18] proposed a method to normalize time domain 
features by upsampling the heart beats. In a similar manner, 
Saechia et al., [19] normalized to healthy durations and then 
divided into three subsequences: P wave, QRS complex, and 
T wave. The Fourier transform was applied on a heart beat 
itself and all three subsequences. 

Zhang and Wei [20] suggested 14 commonly used 
features from ECG heart beats on which a PCA was applied 
to reduce dimensionality. A classification method based on 
Bayes’ Theorem was proposed to maximize the posterior 
probability given prior probabilities and class-conditional 
densities. Singh and Gupta [21] proposed a way to delineate 
the P and T waveforms for accurate feature extraction. 
Boumbarov et al. [22] investigated different models such as 
HMM-GMM (hidden markov model with gaussian mixture 
model), HMM-SGM (Hidden Markov model with single 
Gaussian model) and CRF (Conditional Random Field), 
to determine different fiducial points in an ECG segment, 
followed by PCA and LDA for dimensionality reduction. 
Ting and Salleh [23] described in 2010 a nonlinear dynamical 
model to represent the ECG in a state space form with the 
posterior states inferred by an extended Kalman filter. 

Another fiducial-based method was proposed by 
Tawfik et al. [24]. In this work, the ECG segment between 
the QRS complex and the T wave was first extracted and 
normalized in the time domain by using Framingham 
correction formula or by assuming constant QT interval. 
The DCT was then applied, and the coefficients were fed into 
a neural network for classification. In summary, although 
a number of fiducial-based approaches have been reported 
for ECG-based biometrics, accurate localization of fiducial 
points remains a big challenge. This ambiguity risks the 
accuracy of the respective recognizers which require the 
precise location of such points. In the likely event of failing to 
adequately determine the locations of these points, fiducial 
approaches would rather reject the heart beat and require 
an extra reading, rather than risking the accuracy of their 
decision. This, however, results in increased rejection rates. 


3.2. Fiducial-Independent Approaches. The majority of the 
nonfiducial approaches were reported after 2006. Among 
the earliest is Plataniotis et al’s [25] proposal for an 
autocorrelation- (AC-) based feature extractor. With the 
objective of capturing the repetitive pattern of ECG, the 
authors suggested the AC of an ECG segment as a way to 
avoid fiducial points detection. Wiibbeler et al. [4] have 
also reported an ECG-based human recognizer by extracting 
biometric features from a combination of leads I, II, and 
III, that is, a two-dimensional heart vector also known as 
the characteristic of the ECG. A methodology for ECG 
synthesis was proposed by Molina et al. [26]. A heart beat 
was normalized and compared with its estimate, which was 
previously constructed from itself and the templates from 
a claimed identity. Chan et al. [27] reported ECG signal 
collection from the fingers by asking the participants to 
hold two electrode pads with their thumb and index finger. 
The wavelet distance was used as the similarity measure 
with a classification accuracy of 89.1%, which outperformed 


other methods such as the percent residual distance and the 
correlation coefficient. Chiu et al. [28] proposed the use of 
DWT on heuristically isolated pulses. More precisely, every 
heart beat was determined on the ECG signal, as 43 samples 
backward and 84 samples forward from the R peaks. The 
DWT was used for feature extraction and the Euclidean 
distance as the similarity measure. 

Fatemian and Hatzinakos [29] also suggested the wavelet 
transform to denoise and delineate the ECG signals, followed 
by a process wherein every heart beat was resampled, nor- 
malized, aligned, and averaged to create one strong template 
per subject. A correlation analysis was directly applied to test 
heart beats and the template since the gallery size was greatly 
reduced. 

The spectrogram was employed in [30] to transform the 
ECG into a set of time-frequency bins which were mod- 
eled by independent normal distributions. Dimensionality 
reduction was based on Kullback-Leibler divergence where 
a feature is selected only if the relative entropy between 
itself and the nominal model (which is the spectrogram of 
all subjects in database) is larger than a certain threshold. 
Ye et al. [5] applied the discrete wavelet transform (DWT) 
and independent component analysis (ICA) on ECG heart 
beat segments to obtain 118 and 18 features, respectively. 
The feature vectors were concatenated. The dimensionality 
of the feature space was subsequently reduced from 136 to 
26 using PCA which retained 99% of the data’s variance. 
Coutinho et al. [31] isolated the heart beats and performed 
an 8-bit uniform quantization to map the ECG samples 
to strings from a 256-symbol alphabet. Autoregressive 
modeling was used in [32], and the cepstral domain was 
investigated in [33]. 

It is clear from the above that a large variety of fidu- 
cial-independent techniques have been proposed for ECG 
biometric analysis. While some approaches are more com- 
putationally intensive than others, or they operate on heart 
beats rather than finite ECG segments, there are practically a 
number of open issues in the literature with regard to ECG 
biometrics. Among the most prominent ones is the question 
of signal stability, or permanence, with time. The majority 
of prior works did not examine the evolution of the ECG 
signal with time. To some extent, the sources of intrasubject 
variability of the ECG signal have been ignored. We advocate 
that factors affecting the ECG waveform, which may render 
the biometric template less accurate, should be carefully 
studied and considered to enable real-life deployment of this 
technology. 


4. The Autocorrelation/Linear 
Discriminant Analysis Method 


ECG biometrics is essentially a pattern recognition problem, 
comprised of three distinct steps, that is, preprocessing, fea- 
ture extraction, and classification. 


Preprocessing. The ECG data in raw format contain both 
high (powerline interference) and low frequency noise (base- 
line wander) that needs to be eliminated. Baseline wander is 
caused by low frequency components that force the signal to 
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extend away from the isoelectric line. The source of this kind 
of artifacts is respiration, body movement, or inadequate 
electrode attachment. Powerline interference is generated by 
poor grounding or conflicts with nearby devices. 

To reduce the effects of noise, a butterworth bandpass 
filter of order 4 is used. Based on the spectral properties of 
each wave in the heart beat, the cut-off frequencies of the 
filter are 1 Hz and 40 Hz. The order of the filter and the 
passband frequencies are selected based on empirical results 
[34, 35]. 


Feature Extraction. As mentioned before, the core of the pro- 
posed feature extraction method is the autocorrelation (AC) 
of ECG signals. The rationale for AC is that it captures the 
repetitive property of the ECG signal in a way that only 
significant, iterative components contribute to the waveform, 
that is, the P wave, the QRS complex, and T wave. By 
analyzing the AC, incidental patterns of low discriminative 
power are attenuated, while persistent components of dis- 
criminative power are brought to light. 

The syllogism behind AC with respect to fiducial points 
detection is that it blends, into a sequence of sums of 
products, ECG samples that would otherwise need to be 
subjected to fiducial detection. Furthermore, the AC allows 
a shift invariant representation of similarity features over 
multiple cycles. The AC can be computed as 


N-|m|-1 

> x[i]x[i+m], (1) 

i=0 
where x[i] is the windowed ECG for i = 0,1...(N—|m|-—1), 
and x[i + m] is the time-shifted version of the windowed 
ECG with a time lag of m = 0,1,...(M- 1); M « N. 
Even though the major contributors to the AC are the three 
characteristic waves, normalization is required because large 
variations in amplitudes appear, even among the windows 
of the same subject. In addition, only a segment of the AC 
vector propagates to LDA, as defined between the zero lag 
instance and up to approximately the length of the P wave 
and the QRS complex. This is because these components are 
the least affected by the heart rate variability [7]. 

An AC vector can be used directly for classification. 
However, it is important to further reduce the intrasubject 
variability in order to control false rejection. In addition, 
depending on the sampling frequency of the ECG signal, the 
dimensionality of an AC window can be considerably high. 
For these reasons the linear discriminant analysis (LDA) is 
recruited for dimensionality reduction. 

The LDA is a well-known machine learning method for 
feature extraction. Supervised learning is performed in a 
transform domain so that the AC vector’s dimensionality is 
reduced and the classes are better separable. The remaining 
discussion is based on the following definitions. 


Rxx[m] = 


(i) Let U be the number of classes, that is, the number of 
subjects registered in the system. 


(ii) Let U; be the number of AC windows for a subject 
(class) i, where i = 1,..., U. 
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(iii) We define as z;; an AC window j, where i = 1,..., Uj 
and j = 1,...,U. 

(iv) Let Z; be the set of AC windows for a subject (class) 
i, defined as Z; = tng) as 

(v) Let Z be a training set consisting of all AC windows 
of all subjects, that is, Z = {Zj}¥.). 


Then a set of K feature basis vectors ee aan can be 
estimated by maximizing the Fisher ratio which is equivalent 
to solving the following eigenvalue problem: 

T 
Ly" Soy | (2) 


argmaXx 7- Fra T 
y 


lyTSwy] 


where y = [y1,..., Yg] and S, and S,, are the interclass and 
intraclass scatter matrices respectively, computed as follows: 


§= 5 eames" 
b= ye Z)(zj—Z) , 
(3) 


where Z; = (1/U;) pan Zij is the mean of class Z; and N is 


the total number of windows and N = XY, Uj. 

The maximization of Fisher’s ratio is equivalent to 
forcing large separation between projected ECG windows of 
different subjects and small variance between windows of the 
same subject. The LDA finds y as the K most significant 
eigenvectors of (Sw) Sp which correspond to the first K 
largest eigenvalues. A test input window z is subjected to 
the linear projection y = w/z, prior to classification. For 
the purposes of this work, y is referred to as the biometric 
template. 

It is important to note that ECG biometrics benefit from 
supervised machine learning approaches more than other 
biometric modalities. This is because of this signal’s dynamic 
and time-dependent nature which leads the biometric to 
exhibit higher intraclass variability than traditional biomet- 
ric modalities. With the LDA one can essentially control false 
acceptance and false rejection. 


Classification. For biometric matching, input and gallery 
templates are associated using the Euclidean distance as a 
measure of dissimilarity, while the final decision is made 
upon voting of k-nearest neighbors. The normalized Euclide- 
an distance is computed as follows: 


D(yi,y2) = vo - y2) yı - y2) (4) 


where K is the dimensionality of the feature vectors. For a 
U class problem, LDA can reduce the dimensionality of the 
feature space to U — 1 due to the fact that the rank of the 
between class scatter matrix cannot go beyond U — 1. 


5. Template Destabilization due to Emotions 


Central to the design of a robust biometric solution is the 
study of factors that may affect the biometric signal. In the 


ECG case, such factors may be physiological, psychologi- 
cal, or environmental. Muscle contraction and movement, 
body fluids, and powerline interference are examples of 
environmental factors, that is, whose effect on the signal is 
added after its generation. Typical noise filters, such as the 
Butterworth suffice in addressing such artifacts. 

A healthy physiological change is usually expressed by a 
change in the heart rate. From a medical point of view this is, 
a well-studied effect. From a biometric security perspective 
this variation may be addressed by exploring aspects of the 
signal that remain unaltered, that is, the P wave and the QRS 
complex [7]. For instance, in the AC/LDA approach, only a 
segment of the AC vector corresponding to these waves is 
used for LDA projection. 

However, the effects of psychology on the ECG signal 
are not as well defined. While this signal has been employed 
in affective computing [10, 11, 36-42] in most of these 
cases the heart rate was used in order to determine the 
arousal levels. The ECG waveform (not just the heart rate) 
was also examined for valence classification in [9]. It was 
demonstrated that it is possible to detect specific emotions 
from the ECG signal as long as the emotional stimuli are 
active. (Active stimuli engage the subject in the emotion 
induction process (e.g., video gaming, reading, singing). 
Passive stimuli are presented to the subject without his/her 
direct engagement (e.g., film watching, listening to music)). 

In home telemonitoring environments active arousal is 
unavoidable as emotional stimulation is present in everyday 
activities. It is therefore important to investigate whether the 
emotional changes on the ECG can compromise the respec- 
tive biometric template or not. In [2], it was shown that 
over long recording periods, while subjects are performing 
every-day working activities, the ECG biometric template 
may destabilize, that is, change to a degree that endangers 
security. While the experimental procedures reported in [2] 
were not emotion specific, it was concluded that psychology 
is the underlying cause of variation on the ECG signal. 

The purpose of the present work is twofold: 


(1) to associate, within a monitoring session, instances of 
template destabilization with emotion changes; 


(2) to provide a solution to the above problem based on 
biometric template updating. 


The reader should note that the above two objectives 
do not directly encompass methods to detect or classify 
specific emotions. While experimental procedures have been 
designed in order to trigger specific emotional response, 
determining which emotion is experienced is beyond the 
scope of the current work and has been addressed separately 


[9]. 
6. Emotion Elicitation and ECG Signals 


The following experiment was conducted at the Affect and 
Cognition Laboratory of the University of Toronto. The pur- 
pose was to elicit active mental arousal using a commercial 
video game. 

In practice the experiment attempted to have the play- 
er gradually immersed, by increasingly concentrating in 


order to meet the game requirements. A research-platform 
video game was used, namely, the Cube 2: Sauerbraten 
(http://sauerbraten.org/). A pilot game was built to assist the 
needs of the experiment. The subjects got motivated with 
deception, by letting them know that the purpose of the 
experiment is to measure game completion time. 


All participants were seated in front of a computer screen 
and presented with a short introduction to the video game. 
A five-minute pilot game was played, for the participant to 
learn and be adjusted to the game. During that time, no 
physiological response was monitored. When the subject felt 
comfortable with the process, the ECG sensor was placed 
and the main game was initiated. In total 43 volunteers 
participated in this study. Data from one person were 
discarded due to excessive noise. 


Depending on the familiarity of the subject with game 
playing, the duration of the experiment varied between 20 
and 45 minutes. During the game, ECG was monitored using 
Hidalgo’s equivital sensor, which is portable and wireless. 
Unobtrusiveness was very important for the subjects to be 
naturally immersed in the game. ECG was recorded from the 
chest and digitized at 256 Hz. 


Because of the stochastic nature of the game and the 
unforeseeable order of events that can take place, arousal 
annotation was self-determined. For this reason, a video of 
the player’s facial expressions was captured during the game 
(synchronous to ECG). Upon game completion, the subjects 
were asked to watch a playback video of the game and their 
facial expressions while continuously reporting arousal using 
FEELTRACE [43]. Figure 1 shows an example of such a self- 
assessment video, while Figure 2 illustrates the data labeling 
scheme. 


7. Template Updating 


The underlying idea of the proposed approach is to update 
the ECG biometric template at instances of destabilization 
due to psychological changes. An alternative to this solution 
would be to preenrol a subject with a number of different 
templates along the spectrum of emotional responses. How- 
ever, this is not a practical solution since it is not feasible to 
induce such conditions on demand or ensure that one indeed 
succeeded in doing so. Furthermore, such an approach would 
be very inconvenient for real-world environments. 

In the proposed system, the user can be enrolled once, 
by submitting his/her ECG signal irrespective of the psycho- 
logical status. In return, during normal operation, when the 
system detects that the existing biometric template has been 
destabilized, that is, results in low-quality matching with 
newly captured ECG segments, a new biometric template is 
created to replace the first. Therefore, the objective of the 
subsequent analysis is to update the biometric template at 
instances corresponding to the destabilization or decoherence 
of the correlation scores among consecutive ECG readings 
and the biometric template. 

The evaluation of the coherence of ECG readings has 
been discussed in [2, 3]. The same approach is adopted in the 
current analysis; however the objective of the present work 
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Ficure 1: Game and face video playback, used for self-assessment 
of arousal. 


—5 = Bored 


Arousal intensity 


0 100 200 300 400 500 600 
Time (seconds) 


Ficure 2: Data labeling for the active arousal experiment. The 
FEELTRACE is a continuous arousal indication. 


is to associate instances of destabilization with psychological 
changes. 

The proposed system constructs variable-length accumu- 
lated durations of ECG segments based on some fundamental 
time duration. The accumulated segments need to be 
coherent, that is, exhibit high correlation with each other 
(intraburst correlation). When decoherence is observed, tem- 
plate update is initiated based on the latest ECG segments. In 
practice, there is a tradeoff between frequency of template 
updates and computational complexity. Frequent template 
updating implies accurate tracking of events but increases 
the computational effort. On the other hand, infrequent tem- 
plate updating may cause inadequate system performance in 
terms of increased false rejection. To efficiently address this 
problem, a minimum time duration is forced over which the 
system makes a decision about whether the template needs to 
be updated. In an extreme case one can define this minimum 
duration to be equal to the smallest time resolution in the 
system, that is, the fundamental duration (5 seconds for 
the AC/LDA algorithm). However, this is computationally 
inefficient, and therefore the updating instances need to be 
strategically chosen. 

A variable-length accumulated duration (or burst) is 
constructed by accumulating various fundamental durations 
(for the proposed AC/LDA algorithm, these fundamental 
durations correspond to ECG segments of 5-second length). 
The following iterative description can be made. 

Consider the following fundamental durations {d), 
dz,...}, where each d; corresponds to time duration of 5 
seconds. This duration is chosen to acceptably accommodate 
the time resolution requirement of the AC/LDA algorithm. 
Now suppose that, at the current iteration, the current burst, 
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Deurrenty contains u fundamental durations, that is, Dourrent = 
{dk,...,dk+y-1}. For the subsequent segment, dk+u, the two 
choices are as follows. 


(C1) Add di, to the current burst, forming Dpotential = 
{d,...,dk+,}. Continue the operation with dk+u+1 as 
the next candidate. 


(C2) Reject dk+u, and terminate Deurrent- Reinitialize with 
d+, as the start of a new burst. 


The correlation coefficient is used for the estimation of 
the intraburst coherence and allows the system to decide 
between (C1) or (C2) as follows. 


(1) Compute the correlation profile for Dyotentia relative 
to starting point dx. 


(2) Find the minimum correlation value Cmin over 
Dpotential- 


(3) Compare to a threshold cy, for decision: 


Cı 


> 
Cmin — Cth = 0. (5) 
Cy 


In order to account for buffering problems, along with 
the minimum necessary burst duration, the system also uses 
a maximum duration (e.g., dsizemax) which, when reached, 
forces the system to update the biometric template. With this 
treatment the algorithm can reset. 

For the purposes of the current analysis, the subse- 
quent procedure aims to validate that every coherent burst 
describes a true emotional state. To do that, a measure q; 
is introduced to describe the system’s confidence that burst 
i represents consistently one emotional state. As explained 
earlier in the signal collection section, every ECG window 
x(t) (in the burst 7) is labeled as high (H) or low (L) arousal. 
Let Ni, and Ni be the number of windows marked as high 
and low arousal, respectively. Then q; can be calculated as 


max (Ni, Nj) 


(6) 
Ni + Ni 


qi = 


If template updating succeeds, every burst that is identi- 
fied by the updating algorithm should have a high q;, which 
means that with high confidence each burst corresponds to 
either high or low arousal. 

The reader should note that the purpose of this work 
is not to automatically detect emotional states. Emotional 
confidence is calculated as an a posteriori validation to 
demonstrate the coincidence of instances of emotional 
change and biometric template destabilization. 


8. Experimental Results 


The performance of the proposed algorithm for template 
updating was evaluated on emotion-annotated ECG signals. 
ECG readings are available for the duration of the video 
game, as previously described in Section 6. The first few 
seconds of every reading were used for initialization, that 


is, the design of an initial biometric template. Then in the 
sequel, the correlation of this template with subsequent ECG 
readings was determined and, if not sufficient, the template 
was updated. 

The number of template updates within the 30min 
gaming session varies for different individuals. Subjects that 
were very emotionally consistent required only one template 
update while others updated up to eight times. 

The results are shown in Figure 3 which lists the q 
values (in percentage) for all detected bursts of all subjects 
in the database. Even though this measure relies on the 
subjects’ self-reports and discrepancies are expected, the 
average state confidence for the bursts is 96.47%. This 
performance is illustrative of the accuracy of detecting 
homogeneous emotional states, which leads to successful 
template updating. Now, consider the original problem of 
biometric recognition when a burst is terminated due to 
decoherence the system performs a template update. While 
the above results show that each burst corresponds to either 
high arousal or low arousal, the question that naturally 
arises is the following: do two consecutive bursts correspond 
to opposite arousal labels? In other words, is change in arousal 
the only factor that is responsible for burst termination? 

In practice, the template updating algorithm is not only 
affected by emotional states. A burst may be interrupted due 
to one of the following three reasons. 


(1) A state change, that is, a transition from one psycho- 
logical state to another. 


(2) Buffer overflow, that is, when dsizemax is reached. 
dsizemax can be adjusted according to the require- 
ments of the application environment. For the 
present simulation dsizemax was set to 10 minutes. 


(3) Noise artifacts (e.g., due to a sudden movement), that 
are not sufficiently treated by the filter. 


Nevertheless, a template update is necessary in all cases. 

For every individual in the database, a template is 
updated every time a new burst is detected. To quantify 
the verification accuracy after template updating, the false 
acceptance (FA) and rejection (FR) rates are estimated for 
every individual separately. Table 1 lists the equal error rates 
(EER depicts the error rate at which the probability of false 
acceptance equal that of false rejection) that were achieved 
with this treatment, for all subjects in the active arousal 
dataset. The average equal error rate in this case is 3.96%. 
It should be noted that the baseline system performance that 
is without template updating results in 15% EER. Figure 4 
demonstrates the FA and FR tradeoffs for nine randomly 
picked individuals. In each simulation FA was computed 
with comparisons of the updated template of one individual 
against the remaining subjects in the database. 

A parameter that controls the above analysis is cin, that 
is, the threshold on the correlation coefficient that is used to 
determine burst coherency. As this threshold increases, 
stronger coherence is imposed on the bursts, leading to 
smaller EER, as shown in Figure 5. However, this is achieved 
at the expense of more frequent template updating needed 
(ie., higher cost and complexity). 


Subject number 
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Burst number 


FIGURE 3: Detected bursts and respective emotion consistency (q;) for all subjects in the database (N.B.: the above bursts are not necessarily 
of the same duration, i.e., variable-length durations may be detected). The average state confidence is 96.47%. 


TABLE 1: Equal error rate for each individual in the active arousal database, after template updating. Mean equal error rate is 3.96%. 


Subject EER Subject EER Subject EER Subject EER Subject EER 
1 6% 11 4.3% 21 2.65% 31 5% 41 0.2% 
2 4.2% 12 3.6% 22 3% 32 5.16% 42 0% 
3 8% 13 1.1% 23 5.65% 33 2.65% 43 1.35% 
4 4.4% 14 4.3% 24 3.95% 34 9.2% 

5 4% 15 3.2% 25 3.38% 35 8.9% 

6 0% 16 2.5% 26 4.04% 36 1.85% 

7 0.5% 17 2.75% 27 3.35% 37 7.65% 

8 5.5% 18 4.63% 28 1.46% 38 2% 

9 0.05% 19 4.2% 29 2.75% 39 2.95% 

10 3.8% 20 2.3% 30 0.5% 40 13.34% 


9. Conclusion 


In this work, a novel identity recognition system based on 
the ECG signal has been presented. The proposed biometric 
solution is suitable for welfare monitoring environments 
which require remote, efficient, and continuous authenti- 
cation of the involved parties. In such settings, the ECG is 
typically collected among other vital signals and used for 
diagnosis and treatment decisions. Relying authentication 
on the very same modality not only increases security and 


convenience but also enhances user privacy since no other 
credentials are required to perform this task. 

This technology has significant advantages, most of 
which arise from the fact that ECG is a body-internal 
signal. For instance, ECG-enabled biometric systems can 
automatically and inherently assess the “liveness” of the 
biometric reading. This is not the case with traditional 
biometric modalities (e.g., iris or fingerprint) which require 
additional computational effort to assess the liveness of the 
sensor reading. In addition, ECG is a continuous signal that 
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Figure 4: Verification performance with template updating for 9 individuals of the active arousal database. 
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Figure 5: Equal error rates of the template updating algorithm for various coherence thresholds cn. 
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can be conveniently used to authenticate users several times, 
each time using a fresh reading. 

This paper addressed the problem of intrasubject vari- 
ability of the ECG signal due to psychological changes. It 
was demonstrated that a biometric signature designed at a 
particular time may not allow for robust matching at a 
later time, under different emotional conditions (template 
destabilization). A solution based on template updating 
was proposed for welfare monitoring environments. The 
performance was evaluated over ECG signals from 43 
volunteers, under various arousal conditions induced with a 
video game. On average, an equal error rate of 3.96% was 
achieved, which represents a dramatic reduction from the 
EER of 15% for the particular dataset in the absence of 
template updating. 
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